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ABSTRACT " . ^ ^ . . 

' ' A method of individual clas^r.oom research is- pr6posed 

and demonstrated in this' paper. Its pjir pose was to show that, by 
capitalizing on baseline data collected from a similar prior class 
and^ focusing treatment on difficult ^est items^. the inetiructor can 
dev^op sensitive wi-t&iin-class designs for the detection* Of treatment 
effects. .An experiment using written objectives < gix)sn. to students to, 
aid their learning and using the single classroom design shows that 
objectives significantly improve learning. The experiment was 
aond^ctted with 54 students in^ an ^introductory psyjchology qlaSs at^£he^ 
University of Washington. Students we 3:e given^ lecture objectives but 
1x0 objectives for the reading assignments for the ':^lr^ four wefek 
period. During the second four*' week period a reversal .was made and • 
objectives were given- with the reading^^ assignments but nqt with the , 
lectures. Student^ perf orm:ance- wis measured with multiple choice test* • 
qfiestion#. In ad*d it ion to improving learning^ , the experiment showed 
iAat-^irritten with item difficulty. These results 

lead'to the recommendation that instructors engage in personal 
inquify into their .efforts to influence student learning and th^ta 
such inquiry be added' td the^ educational research lite^ 

(Author/BIX) ^ , ; ' " 
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Absstract 

♦ 

A nethod'of individual Instructor classroom research is proposed 
and denonstrated in this paper By capitalizing on baseline data 
collected from a sixnilar^ prior class and focussing treatment upon 
difficult test Items^ the instructor can develop sensitive vithin ^ 
class designs for the detection. of treatment effects^ An experiment 
using vritten objectives give^i to the students to aid their learning 
and using the single classroom design shows that objectives significantly 
improve learning. Written objeptives are also shown to interact with 
item difficulty. The experiment leads to the recommendation that in- 
structors^engage in personal Inquiry into their efforts to influence 
, Student learning and tl^t such inquiry be added to t:he educational 
reseaxch llterattire. * ^ ^ - 
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Educational Research . , .} 
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Inferential statistics has been developed to solve » In^ a quantitative 
mnner» the^ problems Inherent In generalizing from a specific sanple to an 
abatracjt population. Even wl^h the cautions provided by Inferential 
statlstlcst a conmon error made In their use Is overgenerallsatlon. This error 
suggests that we hold the goal or value *of explaining the vorld' In general too 
dearly and consequently bias our thinking 1k^ the direction of too iiuch a^l^strae-' 
tion. Ih educational research terms this translates into trying 'to find 
instructional methods appropriate in air classrooms for all instructors, 
perhaps this dream Is too grandiose. The individual teacher is primarily con« 
earned with his class, his students, and his ability* to Influence those students 
•in ways he judges to be favorable. . ThQ teacher Is ^gaged in. the* exercise of 
personal Influence and needs a method of inquiry that Xdlll allow hima to assess 
the outcome of his patticular influeuQe attempts, the only igeneraXlza tion he 
fteeds to be concerned with is that of transferring his Impa^ to a i;iew group 
of students. • * • ^ . ' ^ ■ ^ 

Educational research needs €o move to the individual |;lij(structor*s claBS* < 
room if rapid progress on 'theories , of instruction is to bd made, possible. There 
are two reasons for such a move, (a) Tile cost of d<;^ing "fpeiiieaxch with many . 
classrooms, several teachers, and larg§ numbers pf student^ is too great; we 
will never have a large number of -such studies. Thia. appx^qach to Cissearch Is 
surely needed; but because of its limits, effort and oipportiinity, cannoj; be the. 
major source, of educational data. f}>) Tlie shoirf: laboratory experlm^t on the 
other hand should not serve as the major data base fot instructional theory * 
be<:ause of the possible distortions introduced into the data due ta the small 
size and simplicity 1>f the laboratory situation. For example, ^ subject asked 
to learn short passag^es of prose In a one hour period followed ^y immediate 
testing may not engage in the same behaviors he would use in reading a textbook 
over a period of several' weeks. While the laboratory can certainly suggest 
potentially relevant variables. to the individual instructor, the expectation of 
generalization to the classroom, while positive, must remain low until extensive 
dnsroom tests have been made; However, the individual classroom has t^e right 
size^and craiplexlty from the viewpoint of the learning process and is the right 
size from the viewpoint of convenience in the research effort. 



If the individual Instructor Is to be the principal researcher, he needs^ 
a sharper set of research methods than is no.rmai^y provided. The stability of 
Item dlf f icultles^ f rom quarter to quarter and class to class opens the possl* 
bility of improved research^ techniques (AndersQxi^ 1975). In"^ the Anderson ^ 
'study* item difficulty is defined as the proportion of students choosing the 
correct response to test item. Item difficulties were calculated fof 
Identlcial exantinatlons given to students In several sections of a Fortran 
programming class and these item difficulties were. then correlated on ^n item 
by item basi^. The average inter correlation o^^ive classes in the autumn , 
of 1974 was 173 and the average Intercorrelatlon: of eight, classes In. the v^ter 
of 1975 was .71.. These "high correlations were found in s^lte of considerable 
variation in the sections^ e^g*? different instructors, different 'practice 
problCTis, different lectures, etc. These findings indicate that material 
easily learned by one cias'^s will be easily learned by^ another clags and^that^ * 
difficult material la one class will prove difficult In a second class. Be- 
caikae of the commonaXity demonstrated by the high correlations between classes, 
we would expect data from one quartet to be useful as a reliable baseline for . 
data .gathered lii subsequent quaiftei^S. The high correlations encourage the 
subtraction of baseline Item difficulties from item difficulties obtained ' 
dtirlng the- treatment quarter In order to arrive at a change score which more 
sensitively reflects the effect 6f the treatment.^ '.J 

The effect of changing clashes (different subjects) can be ruled out by 
applying the^treateent to part of the to-*be-*leamed matei^ial. This step - 
a4.1ows a within d^s comparison of the change from baseline for untreated Y 

atid treated item difficulties. ^ - ^ 

f* . -•»'•.■ 

One final touch Is needed because the item difficulties for a test given 
in a live classroom tend to form a Skewe^^ dTlstribution with the peak above the 
mean difficulty^ Items which basellhe at, high difficulty values (easy items) 
4o not have much room for change, i.e^ there may be a ceiling effect on the 
majority of the test items. The group of items selected for treatment should 
thus-cottsist of one-half of the items having Xow difficulties in the baseline 
^period. Treatment is confined to one-half of the difficult items because the 
jtemalxilng difficult items are needed as a control for regression effects. 
Basellnlng item difficulties, within class design, and focus of treatment 
analysis* on difficult items should all contribute to tl\e sensitivity of the 



claslsroom design aa a riesearch tool. (These design features strongly refsemble 
the research methods developed by t'he beli^ylor modlfiers,^ Bandura, 1969, land 
their work should provide many useful cues to the classroom Instructor ^gaged 
in Inquiry into hl^ personal effectiveness.) :^ ^ 

. The remainder of this papet illustrates the classroom design features 
Just mentioned in connection with a specific treatment • Duchastel and Iierrill\ 
(1973) reviewed several ^classroom and laboratory studies which assessed the 
effectiveness of written objectives in Improving student performance. ^ Approx-* 
imately 45% of these studies showed objectives to* be ^beneficial. However, 
many|of 'the cpmparlsons made in the classroom were poorly contrdlled, I.e., 
different classes|!^re used for control and treatment, it^ difficulties were 
not used, and thi^e was no focus on difficult items. The purpose^ of this study 
is thus two fold ii (a) to provide a d^onstration of an Individual classroom 
research methodology and (b) to determine; whether. Qr not pr&senting^ students 

with written objectivea will favorably influence their learning. 

• . ■ ■ , ' -f.- - ' ' ' ■ » ■ ■ * . . . ^ ■ ■ - . . 

. ^ ■ . • . / - . , 

. / Methods ' ' 

Subjects » Baseline Item difficulties were collected frdm 64 Introductory 
^^sychology students ^t Shoreline Community College in Seattle, Washington. 
There were tifO separ^ate but similar sections of the introductory class; each 
Section was ^Iven the same tests agd. the data from the two sections was pooled 
for scoring. [ The experiment proper was conducted with 54 students In an evening 
Introductory psychology class at the University of Wa'shing^pn. 

Course wkterlals . The reading assigmirentk for both courses were taken 
figgm Beach, Psychology ; Core Concepts and Special Topiics, 1973, and from a 
book of readings provided with ,the test (Sjursen & Beabh^ Readings In Psychology ; 
Core Concepts and Special Topics , 1973). ' The page ntimbers of the reading asslgn- 
ii^ents were given to the students at least; two weeks prior to the testings dat^s. 
Tiie if'est of the content of the course was presented via lecture^. An effort 
was made to prevent overlap In the content of the readings and lectures and 
the author estimates .that the effort was 85% successful. ' 

Research design . During the first four weeks of the experimental quarter 
(U of V), the students were giyen, at the beginning of each lecture, a mimeo-- 
graphed ^heet w]^ch had statements of, the major points of the lecture written 
as objectives. For i^ample, ^ 



Each student will recognize an , unobtrusive/ measure as 
a peaiurement which does not al'ter real world eventa during . ' 

' - ... " Its use in t^surlng* 

Given a n^w eKperlment testing Bi^rket's ecological 
^ ^ ' theory and the knowledge that the experiment confirmed the 

theory* eact/ student will recognize the outcome of the 
c ^ experiment. . - ^ ^ 

■ . « . . ■ ■ ■ ■ 

The students were given 63 such objectives during tine four week period. No^ 

■ ' " y . . . 

objectives were provided for the reading assignments during this period. In . 

,the second four weeks of r the quarter a jreversal occurred; 50 objectives were 

g^ven with the reading assignments and no objectives i^ere given with lectures # 

There was no experimentation or data analysis during the last two weeks o^ the 

quartei^'Of 'during the week of final examinations, the \ objectives were. written 

Rafter the text and lectures were written and so they served primarily to miake 

elicit the g^als ^f or 'learning Implicitly embodied in the written material. 

Student performance was measure^ with multiple choice test questions. 

e que8t;ions which corresponded to ""the^^xample ojbject Ives are as follows: 

A museum dijtecto^ measures the wear andfteai^ on the 
floor tlies In front of pictures to a^ess their popularity. 



This is an example of... 

A. an unobtrusive measure, ' ' I 

B. operator generated data. ^ 

C. a discriminative stimuli;^ 

D. an experimental ttenlpulation^ ' 

^ Wicker tested Barker's conclusions concerning , the under- 
manhlng and overmanning of behavior settings in small (338 
member) and large^ (1559 member) ^churches. The churches were 
both ia the same lar^e clty» were the same^ denomination ^ and 
had members from the' same socioeconomic class. Bis findings 
agreed with Barker's population studies. Which oi the follow- 
^Ing Is true of Wlaker's findings? 

The large chu|j^h offered a greater variety of . j 
behavior setting. . ^ ! 

The members af |he small church participated in a 
greater varietj^ of behavior settings. 
Small church members donated ihore money^^tjer year« 
All of the above-^. . 
' ' . ' . \ 



A. 
B. 

D. 



The responiaes &£ the experimental Uiilyerslty of Washington ^stpdents were t^irhed^ 
Into an It^i difflctaty (proportion of students restionding; correctly) from which 
the item difficulty for the Shoreline sttt4ehts was subtracjted. .The resulting 



• change scores were separated on two dimensions, (a) presence or absence of ' 
objectives and (b) high (abov^ 60%) or low .(below 59%) item difficulty, and 

^tialyzed with analysis of variance techniques. The high or low difficulty 

* • ' ■ , / . ■ •\ ^ ^ 
split was done using the baseline difficulties collected at Shoreline 

Community College. |^ ^ 

^ / Results ' ' ' * , 

There were 37 test items in cpmmon between the Shoreline and University 

of Washington classes during the first four week period ^ 22 of those related 

to the text and 15 related to the lectures. The 22 items related to the text 

had a Pearson product moment correlation of .83 between^ the two classes whereas 

'the 15 Itemp related to the lectures (objectives preseiur) correlated .52. For 

the second four week period, there were 40 coznmon items^ 22 related to the 

text (objectives present) and 18 to t^e lectures. In this period the lecture 

question, item difficulties correlated .79 and the text question item dlffi** 

■ • ■ ■ ^' - ■ . .* • 

culties correlated .52. Note that where no changes were Intrpduced to the 

course content the correlation of the Shoreline baseline difficulties with the 

University of Washington difficulties is high (.83 and ,79) wheteas change ef- 

forts Idwered tH^ Correlations obtained (.52 a|id .52). Considerable^commonality 

exists in the performance. of these two clashes. ^ ; 

More to the point are the xfesults of a two X twb unweighted means ^analysis 

of variance (Myers, 1972, p. 116) performed on the difference scores. Recall 

that each difference score^wa8 obtained by subtracting the item difficulty of 

the i.^^ item from the Shoreline crass from the item difficulty of± item at 

the University of Washington. Table 1 shpws the mean change scores for each 

cell while table 2 reports the analysis of variance data .o The presence of 

objectives significantly improves student performance, the change scores are 

clearly larger with more difficult items, and the significant objectives X 

difficulty interaction shows that ^he objectives had their biggest impact on 

the dif ficult items. These ei^fect6[ are discoverable with the design' used eved^ 

though the University of Washington studi^nts performed in general at a higher 

level. • " ; '. 
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N, Is the number. of test question change scores 
i/ Included In the- cell. - . 
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Jmalysis of Variance Data • 
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Difficulty . 
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S/0>a X Diff CAdj) 
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73 


.009 
. . .0056 
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.009 • ' 

' .021 
,0056 
.00d79 


11.39** 
** 

26; 58 * 
" 7.09* 



.01 



p < ^001 



Dlacu^Bion 



Tlie methods of experini|»ital analysis used 
of objectives to tnafce a :#lgnlf icant dl^^^ perfortnan^Je of Introductory 

psychoid^ students (p ^ •001) • It Itf Inteteaftlng; to noj^^ that without the 
sensitivity of* the analysis performed these. differences would not have been found 
An independent t test comparljaig the Item difficulties of Items wltli objectives 
with the Item difficulties of the Items wlthftut objectives showed t^o significant 
difference between the two type^ of Items (tyg « l.*02, p > '.fO); The finer 




this study allowed th^ |>resence 



analysis allowed by using baselli^e data frc 



previous classes is cleatly needed 



If we are to assess the i&ff ect of single tr^tment ^variablea .In complex class- 
roon^ situations^ " . / , , , ' . ^ 

The interaction of Itrai dlfflcultjr wijbh the^wrlttea-objectlves treatment ^ 
points to a n^ommon weakness in the design ^f classroom experiments, ^e need to. 
focus our analysis ou the more sensitive ^est Items if we are ^o detect the full * 
effect of ojur treatment. ^ Ceiling effects work against the detection of differ- 
ences between the treatment and a ppntrol condition and lead us to conclude, 
that our treatment is not effective when In fact It may be quite effective. 

^Thls interaction clearly points to the need for Instruct Wal :bQprovement 
effbirts which focus on selected segments ok course content ^ and so we are led 

\to the same focus on difficult material In the classroom that pair edrassoclate 



IC? 



researchers (Atkln^on» J.972; Atklnsoit & Paulson » 1972) have used In laboratory 

studies-;' ' ; -i ^ •. ^- ' . ' ^ 

« .1 There are limitations to th^s type of .^tudy. Will the results generalize? 

It Is possible that thie author has 4 talent for writing objectives which other 

instructor's dp not share/ but that does „ not seem likely; The obj|,ectives are 

nothing more , than a , statement of the key concepts the students aire to 4earn. 

P^^rhaps the effect ot objectives discovered here ^ is partily due to /the nature 

of Intrdductory psychology or the multiple dholce testing format* Until this. 

■ ■ ■ * ' . ■ ■ . 

experiment is conducted in other classrooms # with different content and 

d^ifferent testing methods $ the^ question/ '^Will the results generalize?",. cannot 

be ri^rously answered. Certainly the results of jbhis one experiment should 

lead to the positive expe ct^tion that otiject^ves ^ox any other Variable ptoduc- 

ing positive results in the class|;oom)^ make a ^difference elsewhere, A 

second problem concerns the Inabjlll^ of the single c):a8sroom, design to introduce 

variability on diW^idions which tiiay b6 relevant to student p^^rmance. For 

example, the individual instructor would ^lormally be jse^n, as^ havii^g a siii^le^ 

unvariable cOhst^.lation of personality l^r^lts. Research with,varlii[bles llke^ ' 

personality will nave to ^^vdlve more than one ins true tar • . ^ . / ^ 

. All activities engaged in by humans are at some ftindpfin^tal level 'lie^sondl.^ 

The individual instructor teaches liis students, ^his course and this personal ^ ; 

environment is where the i\nistructor must strive to improve his ability to teach* 

If he wants' to be systematic abotit such Improvement efforts, he need& research 

designs which allow personal inqOlry* General izaWoA to bther instructor's 

classes, while it may occur,* is nbt the issue » In making the individual classy 

room the locjation of emphasis in the educationfil research effort, we are ^ 

focussing on the one^place where resfea^ch cati find^ vigorous home. The , timber 

of opportui&ities for 'such research is. large and the effort needed is of ten no 

more than the effort teachers normally expend to impi^ove thei^ courses; Evehtually 

enough single ^c§se' experiments such as the experiment ^epdrted here will accumulate 

so that generalizations may ^pear. Even if the research j^tcomes prove <situa- ' , 

tion specific, the methods used to discover the partiCMlar results will, generalise* 

The destructor need only adapt the reseatch methods to his personal inquiry. 
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